A Note on the PAC Bayesian Theorem
نویسنده
چکیده
We prove general exponential moment inequalities for averages of [0,1]valued iid random variables and use them to tighten the PAC Bayesian Theorem. The logarithmic dependence on the sample count in the enumerator of the PAC Bayesian bound is halved.
منابع مشابه
The Proof of McAllester’s PAC-Bayesian Theorem
Extends McAllester’s PAC-Bayesian theorem [4], allowing for other tail bounds than Hoeffding’s, and simplifies the proof somewhat. 1 McAllester’s PAC-Bayesian theorem McAllester’s PAC-Bayesian theorem is an astonishing result. Although its proof requires only elementary techniques, it can be used to obtain very tight generalization error bounds for the Gibbs versions of (approximate) Bayesian c...
متن کاملPAC-Bayesian Theory Meets Bayesian Inference
We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam’s razor criteria, under the assumption that the data ...
متن کاملSupplementary Material to A PAC-Bayesian Approach forDomain Adaptation with Specialization to Linear Classifiers
In this document, Section 1 contains some lemmas used in subsequent proofs, Section 2 presents an extended proof of the bound on the domain disagreement disρ(DS , DT ) (Theorem 3 of the main paper), Section 3 introduces other PAC-Bayesian bounds for disρ(DS , DT ) and RPT (Gρ), Section 4 shows equations and implementation details about PBDA (our proposed learning algorithm for PAC-Bayesian DA t...
متن کاملNumerical Meshless Method in Conjunction with Bayesian Theorem for Electrical Tomography of Concrete
Electric potential measurement technique (tomography) was introduced as a nondestructive method to evaluate concrete properties and durability. In this study, numerical meshless method was developed to solve a differential equation which simulates electric potential distribution for concrete with inclusion in two dimensions. Therefore, concrete samples with iron block inclusion in different loc...
متن کاملOn PAC-Bayesian Margin Bounds
Over the past few years, progress has been made in obtaining dimension independent margin bounds. In this note, we revisit the PAC-Bayesian margin bounds proposed by Langford and Shawe-Taylor [4] and later refined by McAllester [6]. In addition to simplifying some of the existing arguments, we use a tighter tail bound on the normal distribution to give an explicit margin bound that is a mild va...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره cs.LG/0411099 شماره
صفحات -
تاریخ انتشار 2004